Papers by Viktoriia A. Chekalina
Acceleration of Backpropagation in Linear Layers of Transformer Models Based on Gradient Structure (2026.eacl-srw)
Copied to clipboard
| Challenge: | a sparsity-exploiting backward pass is a memory-efficient way to accelerate LLM fine-tuning. |
| Approach: | They propose a method that exploits padding-induced gradient sparsity to accelerate backward computation. |
| Outcome: | The proposed method achieves a backward pass speedup of 2.15x on GLUE and 1.99x on reasoning benchmarks while maintaining memory usage identical to the regular PyTorch fine-tuning. |
Fast and Accurate Fisher-Guided Quantization via Efficient Kronecker Factorization (2026.acl-long)
Copied to clipboard
| Challenge: | Quantization has shown strong results in preserving model quality under compression, but under aggressive bit-width reductions, even quantization may require additional information to prevent performance degradation. |
| Approach: | They propose a Kronecker-factored approximation that captures second-order curvature information, captured by the Hessian, to achieve a 10 speedup over prior approaches. |
| Outcome: | The proposed method significantly accelerates the most expensive component in second-order quantization – Hessian parameterization . it achieves up to a 10 speedup over prior approaches. |